A fundamental matrix for regular semi-Markov processes

The KTH Visit in Semi-Markov Processes

Semi-markov Decision Processes

Considered are infinite horizon semi-Markov decision processes (SMDPs) with finite state and action spaces. Total expected discounted reward and long-run average expected reward optimality criteria are reviewed. Solution methodology for each criterion is given, constraints and variance sensitivity are also discussed.

Quantum semi-Markov processes.

We construct a large class of non-Markovian master equations that describe the dynamics of open quantum systems featuring strong memory effects, which relies on a quantum generalization of the concept of classical semi-Markov processes. General conditions for the complete positivity of the corresponding quantum dynamical maps are formulated. The resulting non-Markovian quantum processes allow t...

Semi-Markov Decision Processes

The previous chapter dealt with the discrete-time Markov decision model. In this model, decisions can be made only at fixed epochs t = 0, 1, . . . . However, in many stochastic control problems the times between the decision epochs are not constant but random. A possible tool for analysing such problems is the semiMarkov decision model. In Section 7.1 we discuss the basic elements of this model...

the kth visit in semi-markov processes